Learning with Partial Observations in General-sum Stochastic Games

In many situations, multiagent systems must deal with partial observability that agents have in the environment. In these cases, finding optimal solutions is often intractable for more than two agents and approximated solutions are often the only way to solve these problems. The models known to represent this kind of problem is Partially Observable Stochastic Game (POSG). Such a model is usually solved by approaches where the environment dynamic is known. In this article, we use the observability itself as a way to approximate the multiagent solution in cases where agents learn using interaction an environment of which the dynamic is not known. We restrict the notion of observability to mutual observability where each agent can see a subpart of the environment and all agents together have a complete perception. We present a class of reinforcement learning algorithms based on equilibria solutions and show how those solutions can be approximated on some single step games and on more general stochastic games. We present results using different known equilibria (Nash and Correlated) and show that limited neighbourhood allows obtaining either equilibrium solutions or approximated equilibrium solutions when computing only equilibria with limited number of agents.